Conventionally, various methods have been proposed which display 3-dimensional images. Of these, “binocular methods” using binocular parallax are generally used. Specifically, a stereoscopic view is achieved by providing left and right-eye images having binocular parallax and projecting them separately on the left and right eyes, respectively.
FIG. 16 is a conceptual view for illustrating a “alternating-field system” as one of the typical binocular methods.
In this alternating-field system, the left-eye image and right-eye image are interlaced on alternate horizontal lines of one pixel as shown in FIG. 16, so that the left-eye image and right-eye image will be switched and displayed alternately. The left-eye image and right-eye image therefore have half the vertical resolution compared to that in normal 2-dimensional display mode. An observer should put on shutter glasses that open and close in synchronism with the switching period of the display. The shutter used here opens the left-eye side and closes the right-eye side when the left-eye image is displayed and closes the left-eye side and opens the right-eye side when the right-eye image is displayed. With this arrangement, the left-eye image is observed by the left eye alone while the right-eye image is observed by the right eye alone, to achieve stereoscopic view.
FIG. 17 is a conceptual view for illustrating another typical scheme of the binocular methods, namely “parallax barrier system”.
FIG. 17(a) is a view showing the principle of the cause of parallax. FIG. 17(b) is a view showing an image frame displayed in the parallax barrier system.
In FIG. 17(a), an image in which the left-eye image and right-eye image are interlaced on alternate vertical lines of one pixel as shown in FIG. 17(b), is displayed on an image display panel 401 while a parallax barrier 402 with slits having a slit width smaller than the interval between the pixels for an identical viewpoint is placed in front of image display panel 401, whereby the left-eye image is observed by the left eye 403 alone while the right-eye image is observed by the right eye 404 alone, to achieve stereoscopic view.
Incidentally, there is another method, the “lenticular system” for achieving 3-dimensional display of an image as shown in FIG. 17(b), which is similar to the parallax barrier system. One example of a recording data format used in the lenticular system is disclosed by Japanese Patent Application Laid-open Hei 11-41627.
FIG. 18 is a conceptual view showing one example of a recording data format of the lenticular system. A left-eye image 501 as shown in FIG. 18(a) and a right-eye image 502 as shown in FIG. 18(b) are each thinned to half with respect to the horizontal direction, forming and recording a frame of complex image 503 as shown in FIG. 18(c). Upon reproduction, this complex image 503 is rearranged to form a composite image as shown in FIG. 17(b).
Although not limited to 3-dimensional images, Japanese Patent Application Laid-open No. 2001-337994 discloses a method of storing additional information for identification of a thumbnail image and displaying the additional information laid over the thumbnail image on a display device.
In the above way, in the method of achieving stereoscopic view by letting the left and right eyes observe different images, it is possible to practice comfortable stereoscopic view when the distance of counterpart points of the left and right images (will be called parallax, hereinbelow) falls within a certain fixed range. However, as the parallax becomes greater, the images for both eyes will not merge into a stereoscopic view. The magnitude of the parallax at that point, has been reported by, for example “a tentative plan of guidelines for 3-dimensional images” published in 2002 by The Mechanical Social Systems Foundation.
Japanese Patent Application Laid-open 2000-78615 and Japanese Patent Application Laid-open Hei 10-221775 disclose methods of achieving easy display of a 3-dimensional image, when its binocular images are hard to merge into a stereoscopic view due to the magnitude of the above parallax, by shifting the displayed positions of left and right images on a stereoscopic display so as to adjust the parallax.
As stated above, in conventional 3-dimensional display systems, recording of data is done in a fixed recording data format so as to be suited to the display scheme determined on the playback apparatus side, hence no consideration has been taken to make recording data versatile.
A 3-dimensional display involves various necessary information such as the method of thinning of image, the number of viewpoints in a so-called “multi-view scheme” and the like other than the display scheme, these information are not recorded as the recorded data when a single display scheme is used. It is true that if only one identical display scheme is always used, it is not necessary to record these information at all, but the versatility of recording data is markedly reduced because of this. Just referring to the limited cases where data for the parallax barrier system (or the lenticular system) is to be recorded, the left-eye image and right-eye image may be recorded as separate sequences, the data may be recorded as a mixed image in which the left-eye image and right-eye image are arranged horizontally half-and-half in one frame as shown in FIG. 18(c), or the data may be recorded as a composite image in which the left-eye image and right-eye image are interlaced on alternate vertical lines of one pixel as shown in FIG. 17(b). Naturally, data of different recording formats should be handled by different displaying processes, but since it is impossible to know the format of data from the recorded data, there is a problem in that it is impossible to know how the data should be processed for display when a third person gets the data.
Further, in the prior art, no consideration has been taken for recording image data from different viewpoints independently from each other so as to facilitate readout and reproduction of a desired viewpoint image only.
In the prior art, no sufficient consideration has been given to interchangeability with existing apparatus, either. Specifically, in a system disclosed in Japanese Patent Application Laid-open 2001-337994, the display systems capable of interpreting additional information alone have been handled, but the additional information is not useful for display systems that cannot interpret it.
Moreover, when a 3-dimensional image based on the above prior art is enlarged or reduced, the amount of protrusion and the amount of depth of the 3-dimensional image change, hence there occurs a problem that a desired stereoscopic effect cannot be obtained.
Referring first to FIGS. 39 and 40, description hereinbelow will be made briefly on the principle of a stereoscopic display for presenting a stereoscopic view by displaying separate images for the left and right eyes. Both of these drawings are schematic top views showing cases where a user having a binocular distance d is observing a stereoscopic display 1.
Generally, suppose that d[m] represents the distance between eyes of a user, D[m] the distance from the user to stereoscopic display 1, W[m] the width of the display, P[dot] the resolution of the display and l (alphabetical letter l) [dot] the distance between left and right counterpart points of a 3-dimensional image.
Then, the amount of protrusion z[m] when a 3-dimensional image protrudes forward is given byz=(l×W/P)×D/(d+(l×W/P))  Eq. (1)
The amount of depth z[m] when a 3-dimensional image sets back is given byz=(l×W/P)×D/(d−(l×W/P))  Eq. (2)
The parallax θ is given byθ=tan−1(1/2D)×2  Eq. (3)
With this stereoscopic display, when a 3-dimensional image is enlarged or reduced, the extent of disparity between the left and right images changes, hence the resultant image changes in stereoscopic effect. This will be described referring to the 3-dimensional image before enlargement in FIG. 39(a) and the 3-dimensional image after enlargement in FIG. 39(b). When a 3-dimensional image having a protrusion forwards from the stereoscopic display as shown in FIG. 39(a) is enlarged, the amount of protrusion becomes greater as shown in FIG. 39(b). Here, l′ represents the left and right counterpart points after enlargement and z′ the amount of protrusion after enlargement.
On the other hand, when a 3-dimensional image having an setback interior-ward from the stereoscopic display as shown in FIG. 40(a) is enlarged and displayed, the amount of depth becomes greater, and with some magnification ratio, it becomes impossible to present a stereoscopic view because the views of the left and right eyes do not fuse. In contrast, when a 3-dimensional image is reduced in size, the disparity between the left and right images becomes smaller, the amount of protrusion or the amount of depth becomes smaller, presenting a weak stereoscopic effect.
In this way, when a 3-dimensional image is enlarged or reduced, the stereoscopic effect varies as that protrusion becomes greater because the parallax becomes greater when the image is enlarged and conversely depth becomes smaller because the parallax becomes smaller when the image is reduced. Therefore, if a 3-dimensional is enlarged or reduced in the same manner as a usual 2-dimensional image, there occurs a problem that a desired stereoscopic view cannot be obtained causing confusion or the uncomfortable stereoscopic view causes a strain on the eyes.
The present invention has been devised in order to solve the above problems, it is therefore an object of the present invention to provide a 3-dimensional image creating apparatus which can make the image data for 3-dimensional display versatile and permits efficient selection of an arbitrary viewpoint image as well as providing a 3-dimensional image reproducing apparatus for reproducing the data.
It is another object of the present invention to provide a 3-dimensional image processing apparatus, a 3-dimensional image processing program and a recording medium recorded with the program, which can give warning to the user and make correction so as to provide a comfortable stereoscopic view when the parallax quantity varies due to enlargement or reduction in size of a 3-dimensional image so that there is a fear that it becomes difficult to obtain a stereoscopic view or the stereoscopic effect.